Papers with text quality evaluation
Improving Preference Extraction In LLMs By Identifying Latent Knowledge Through Classifying Probes (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are often used as automated judges to evaluate text, but their effectiveness can be hindered by unintentional biases. |
| Approach: | They propose to use linear classifying probes to access LLMs’ latent knowledge and extract more accurate preferences by leveraging differences between contrasting pairs of prompts. |
| Outcome: | The proposed method outperforms generation-based judgements while maintaining similar computational costs. |
Evaluating LLMs’ Assessment of Mixed-Context Hallucination Through the Lens of Summarization (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models generate coherent text and follow instructions across diverse tasks, but a critical challenge in scaling LLM applications is hallucination, where the generated content lacks factual grounding or deviates from the intended discourse context. |
| Approach: | They use summarization as a representative task to evaluate LLMs' capability in detecting mixed-context hallucinations, specifically distinguishing between factual and non-factual hallucinos. |
| Outcome: | The proposed model distinguishes between factual and non-factual hallucinations, and their performance bottlenecks. |